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Objective: To determine whether congenitally deafened adults 
achieve improved speech perception when auditory and visual 
speech information is available after cochlear implantation. 
Study Design: Repeated-measures single subject analysis of 
speech perception in visual-alone, auditory-alone, and audio- 
visual conditions. 

Setting: Neurotologic private practice and research institute. 
Subjects: Eight subjects with profound congenital bilateral 
hearing loss who underwent cochlear implantation as adults 
(aged 18-55 years) between 1995 and 2002 and had at least 1 
year of experience with the implant. 

Main Outcome Measures: Auditory, visual, and audiovisual 
speech perception. 

Results: The median for speech perception scores were as fol- 
lows: visual-alone, 25.9% (range, 12.7-58.1%); auditory-alone, 
5.2% (range, 0-49.4%); and audiovisual, 50.7% (range, 16.5- 
90.8%). Seven of eight subjects did as well or better in the 
audiovisual condition than in either auditory-alone or visual- 



alone conditions alone. Three subjects had audiovisual scores 
greater than what would be expected from a simple additive 
effect of the information from the auditory-alone and visual- 
alone conditions alone, suggesting a superadditive effect of the 
combination of auditory-alone and visual-alone information. 
Three subjects had a simple additive effect of speech per- 
ception in the audiovisual condition. 

Conclusion: Some congenitally deafened subjects who un- 
dergo implantation as adults have significant gains in speech 
perception when auditory information from a cochlear implant 
and visual information by lipreading is available. This 
study shows that some congenitally deafened adults are able 
to integrate auditory information provided by the cochlear 
implant (despite the lack of auditory speech experience before 
implantation) with visual speech information. Key Words: 
Adults — Cochlear implants — Deafness — Lipreading — Speech 
perception. 
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Increasing numbers of congenitally and early deafened 
adults are presenting for evaluation for cochlear implan- 
tation. Why? Satisfied implant users are recommending 
implantation to their deaf friends. The Deaf community 
is becoming more open to cochlear implantation. Pub- 
licity has increased. Availability has increased. 

Cochlear implantation is an effective treatment of 
deafness, but the degree of benefit is highly variable 
among individuals. Greater residual hearing, shorter 
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duration of deafness, and younger age at the time of 
implantation are among the best predictors of superior 
postimplantation speech perception. Reports of early 
generation cochlear implants showed that most prelin- 
gually deafened subjects who underwent implantation as 
teenagers or adults rarely obtain significant improvement 
in open- set speech recognition (1-3). Commonly, the de- 
gree of benefit from a cochlear implant is defined simply 
by auditory sentence recognition testing. Therefore, one 
could conclude that prelingually deafened adults do not 
obtain significant benefit from the cochlear implant (4,5). 
On the basis of this reasoning, these patients are discour- 
aged from undergoing implantation in many implant 
centers. In addition, because many studies report lack of 
significant improvement in speech recognition, payment 
for this expensive intervention is at risk of being denied 
to this group of patients in the current health care cli- 
mate in the United States. 

With new-generation cochlear implants and advance- 
ments in speech processing technology, the best cochlear 
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implant users are reaching high levels of open-set speech 
recognition. Prelingually deafened adult implant can- 
didates might also benefit from these developments. 
Manrique et al. (6) demonstrated the capability of pre- 
lingually deafened adult implant recipients to recognize 
speech signals using the Nucleus 22 (Cochlear Corpo- 
ration, Sydney, Australia). Waltzman and Cohen reported 
improved open-set speech perception in four prelin- 
gually deafened subjects who underwent implantation 
between ages 11 and 20 years using the Continuous 
Interleaved Sampler speech processing strategy (4). This 
was an encouraging improvement in comparison with 
their study of patients with the multichannel Nucleus 
device using early speech processing strategies wherein 
there was no statistically significant difference between 
preoperative and postoperative speech recognition tests 
(3). In a later study, Waltzman et al. (7) evaluated open- 
set speech perception in 14 adult prelingually deafened 
subjects with up to 3 years of postoperative experience 
with the Nucleus CI24M (Cochlear Corporation, Sydney, 
Australia) and Clarion cochlear implants (Advanced 
Bionics Corporation, Sylmar, CA, U.S.A.) using Ad- 
vanced Combination Encoding, Spectral Peak, and Con- 
tinuous Interleaved Sampler speech processing strategies. 
A majority of subjects showed improvement in each of 
the open-set tests used. Fifty percent of subjects showed 
improvement in open-set City University of New York 
sentence recognition, with scores increasing between 
2% and 98%. In Schramm et al.'s (8) case study of 
15 prelingually deafened adult patients with 12 months' 
follow-up after cochlear implantation, open- set Central 
Institute for the Deaf (CID) Everyday Sentences sen- 
tence recognition improved in 9 of 15 subjects. Post- 
implantation scores on CID testing for all patients ranged 
from 0% to 98%, with nine patients showing improve- 
ment compared with preoperative scores (with a range of 
0-75%) and six patients scoring more than 40%. Of the 
six patients who had no correct responses preoperatively 
on the CID test, three scored significantly better post- 
operatively, with scores between 13% and 58%. 

Although these studies are encouraging, auditory-only 
speech perception testing might not adequately reflect 
the benefit a prelingually deafened adult could obtain 
from an implant. It is well known that sighted listeners 
use visual information to supplement and enhance hear- 
ing, especially in challenging situations. Visual speech 
perception is an important component of the speech and 
language skills of hearing-impaired individuals as well. 
In the case of a profoundly hearing impaired individual, 
visual cues are the most important source of language 
information. Deaf individuals can be highly accurate 
visual speech perceivers (9). 

Studies have shown that children with cochlear im- 
plants do benefit in the audiovisual (AV) condition com- 
pared with the auditory-alone (A) and visual-alone (V) 
conditions (10). The question for this study was whether 
individuals with extremely limited auditory experience 
are able to integrate auditory information from the 
cochlear implant with the visual speech information. In 



this study, we evaluated the ability of eight congenitally 
deafened adult cochlear implant recipients to integrate 
auditory and visual information. The hypothesis that was 
tested was that congenitally deafened cochlear implant 
users can identify more words when presented in AV 
speech than in A or V speech alone. 

PATIENTS AND METHODS 
Subjects 

The subjects were chosen from a review of all cochlear im- 
plant patients at the House Ear Clinic. Fifty-four patients with 
congenital or early-onset deafness were identified. Twenty-six 
patients met initial enrollment criteria. All patients who met 
the enrollment criteria for the study were invited by letter to 
participate. The enrollment criteria included 1) congenital pro- 
found sensorineural hearing loss (hearing loss identified before 
the age of 2 and thought to be congenital); 2) cochlear im- 
plantation as adults (age 18-55 years) between 1995 and 2002 
at the House Ear Clinic; 3) current user with at least 1 year of 
experience using the implant; and 4) use of English, Signed 
English, and/or American Sign Language as their native lan- 
guage^). Eight subjects agreed to participate. Eighteen sub- 
jects were not enrolled either because they did not respond to 
the invitation or a secondary review found them to fail the 
enrollment criteria. Of the eight subjects studied, the cause of 
hearing loss was hereditary in three cases, maternal rubella in 
two cases, and unknown in three cases. Seven subjects had a 
lifetime experience with hearing aids and the remaining subject 
had only 1 year of experience with a hearing aid. All of the 
subjects were highly motivated research subjects with college 
educations. All of the patients were full-time users of their 
devices. Table 1 lists further demographic information. 

Methods 

The subjects answered a questionnaire regarding hearing his- 
tory and communication practices. Clinical histories were 
obtained by review of records and by personal interviews. 
Before testing, aided thresholds to warble tone stimuli were 
obtained. Each subject's processor was assessed for appropri- 
ateness of map and sensitivity. The maps were not manipulated 
before testing. Each subject started their test session with 
a fully charged processor. Nucleus 24 users were provided with 
new AA batteries and Clarion users were provided with a fully 
charged battery pack. 

A visual acuity of 20/30 or better on a standard Snellen chart 
was required for inclusion. Each subject completed a comput- 
erized lipreading screening test. The lipreading screening test 
is composed of 30 videorecorded sentences (11,12) from the 
list of CID Everyday Sentences (13). Half of the sentences 
were spoken by a male talker. Participants were tested individ- 
ually in a quiet room. The prerecorded sentence stimuli stored 
on DVD were displayed on a 14-inch color monitor approx- 
imately 0.5 m from the participant. A videodisk player was 
controlled by a personal computer. The personal computer was 
used to present stimuli and to record responses. Playback re- 
sponse collection was controlled by custom computer software. 
Participants viewed each sentence and then typed at a computer 
terminal what they thought the talker had said. Passing scores 
were based on normative data collected for the relevant par- 
ticipant group. 

Speech perception testing was performed in A, V, and AV 
conditions. Testing was performed in a sound-treated booth. 
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ACE, advanced combination encoders; ASL, americal sign language; AU, binaural; CIS, continuous interleaved sampling; Contra, nonimplanted 
ear; HINT, hearing in noise test; Ipsi, implanted ear; N24, Nucleus 24 (Cochlear Corporation, Australia); PTA, Four-frequency (500, 1,000, 2,000, and 
3,000 Hz) average bone conduction threshold; SE, signed english; SPEAK, spectral peak strategy. 



Speech was presented at 65 dB sound pressure level in the 
sound field. The materials were presented via a BETACAM-SP 
recording of a female speaker with the face filling the display 
of a 14-inch Sony Trinitron monitor. One hundred fifty IEEE 
sentences (14) were chosen and categorized into three equiv- 
alently difficult sets based on previous testing in the visual-only 
condition on normal hearing subjects. The average number of 
words per sentence was eight. Analysis of variance showed that 
the three sets of 50 sentences (for the A, V, and AV subtests) 
were not different based on the percentage correct scores of 
normal hearing subjects. For this study, the order of testing for 
the A, V, and AV subtests was determined randomly. Each sub- 
ject was given up to 10 practice sentences immediately before 
the administration of each subtest. 

After being presented with the stimulus sentence, the subject 
typed on a computer keyboard the whole sentence or any 
words or parts of words they understood. They were instructed 
to guess if needed. They had as much time as needed to type 
their answers and could make changes before submitting the 
response for each sentence. For all sentence testing, unambigu- 
ous typographic errors were corrected before scoring. A custom 



software program scored the number of correct words per 
sentence. The software uses a recursive algorithm to find the 
best word level match between a stimulus and response sentence. 

RESULTS 

Lipreading Screening 

The lipreading screening scores are listed in Table 2. 
For deaf subjects, 86 is the lowest passing score. It re- 
presents one-half standard deviation below the mean of 
a normative group of deaf lipreaders. With one exception, 
subject 4, the subjects were expert lipreaders. In general, 
subjects who had high scores on the lipreading screening 
test had high scores on the visual sentence subtest. 

Speech Perception 

The percentages of words correct for the eight 
subjects are presented in Table 2 and Figure 1 (arranged 
from lowest to highest AV score). The median for the 



TABLE 2. Speech perception, lipreading screening, and audiovisual tests 
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A lipreading score of 86 is considered a passing score for deaf subjects. 

A, auditory-alone; AV, audiovisual; HINT, hearing in noise test; V, visual-alone. 
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FIG. 1. A, V, and AV sentence testing 
with percentage of words correct. The AV 
condition is labeled with regard to the effect 
of combined condition: No effect, AV = V; 
additive, A + V = AV; superadditive, AV > 
A + V. 



Subject 4 Subject 6 Subject 3 Subject 7 Subject 1 Subject 8 Subject 2 Subject 5 



sentence testing were as follows: V, 25.9% (range, 12.7- 
58.1%); A, 5.2% (range, 0-49.4%); and AV 50.7% 
(range, 16.5-90.8%). Four subjects had better than 50% 
words correct in the AV condition. The three subjects 
who scored less than 1% words correct in A had the low- 
est words correct in the AV condition. The three subjects 
who scored between 4% and 11% in A (subjects 2, 7, 
and 8) scored 48% or more in AV. As a group, the eight 
subjects performed significantly better in the AV 
condition than in either A (p = 0.008) or V (p = 0.016) 
conditions (Wilcoxon signed ranks test). 

Three subjects (subjects 1, 5, and 6) had a simple 
additive effect of speech perception in the AV condition 
where the percentage of words correct in the AV con- 
dition approximated the percentage of words correct 
obtained by adding the A and V scores (AV = A + V). 
Three subjects (subjects 2, 7, and 8) had AV scores sub- 
stantially greater (by more than 10%) than what would 
be expected from a simple additive effect, suggesting 
a superadditive effect (AV > A + V) of testing in the AV 
condition compared with testing A and V separately. 
Two subjects (subjects 3 and 4) had scores in the AV 
condition very close to the V condition, suggesting no 
effect of the addition of auditory information. 

Subjectively, six of eight subjects reported better 
speech perception with the cochlear implant. Subjects 2 
and 8 did not feel their speech perception was improved 
with the implant, despite understanding more than 50% 
of words in the AV condition. 

Relative Benefit 

A method of calculating relative benefit from the 
addition of a second mode of information was described 
by Lachs et al. (15). This method is helpful for as- 
sessing the gain in accuracy in the AV condition relative 
to the accuracy in a unimodal condition normalized to 
the amount of improvement that is possible based on the 
unimodal score. The relative gain in speech perception 



attributable to the addition of auditory information over 
the V condition can by calculated by the following 
formula: R v = (AV - V)/(100 - V), where AV is the 
speech perception score in the audiovisual condition and 

V is the score in the visual-only condition. 

For the subjects in this study, the median R v is 34%, 
with a range of —2 to 78%. Individual scores are listed 
in Table 2. All but one subject had a positive gain. Five 
subjects had relative gains in the AV condition over the 

V condition greater than 30%. Two subjects showed only 
a small positive gain. 



DISCUSSION 

Despite highlighting definite improvement in speech 
recognition in a proportion of prelingually deafened 
adult cochlear implant patients, some previous studies 
conclude that in most patients the effect is limited and in 
some patients there is no benefit (2-5). However, auditory- 
alone word and sentence recognition scores are not ex- 
clusively representative of how much a patient benefits 
from a cochlear implant. 

Several authors have highlighted significant improve- 
ments in auditory skills, including detection and iden- 
tification of environmental sounds, improvement of 
lipreading, improvement in speech production, and im- 
provement in self-reported quality of life in prelingually 
deafened adult cochlear implant recipients (1 — 4,16). Sus- 
tained use of the implant may also be a good indicator of 
subjective benefit. Even with the earliest devices, some 
patients benefited. Eisenberg (16) reported that 8 of 12 
prelingually deafened adult implant patients in her study 
used their implants daily and were able to enjoy music 
and hear horns and the calling of their names. Some of 
her patients also reported that they were more indepen- 
dent an social and less lonely (16). In Waltzman et al.'s 
article, all 14 subjects continued to use the device on a 
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regular basis and reported that their communication 
skills were enhanced by the implant, including the sub- 
jects who did not show improvement on speech 
recognition scores (7). On the basis of results from the 
Performance Inventory for Profound and Severe Loss 
questionnaire, 7 of 15 subjects in a study by Schramm 
et al. (8) (who obtained scores ranging from 0-41% on 
CID sentences) reported improvement in understanding 
of environmental cues and understanding speech with no 
visual cues. In Zwolan et al.'s (1) study, 11 of 12 sub- 
jects expressed some degree of satisfaction and 9 indi- 
cated by questionnaire that the device improved their 
lipreading ability. 

One reason for apparent discrepancies between speech 
recognition testing and patient-reported outcomes may be 
the limitations of auditory-only speech perception testing. 
Normal hearing and hearing-impaired adults integrate 
auditory and visual cues to enhance speech perception, 
especially under difficult listening conditions (17-20). 
Prelingually deafened children with cochlear implants 
are able to integrate visual cues with auditory in- 
formation provided by the cochlear implant (10,15). 

The gain in speech understanding in the audiovisual 
mode for adult postlingually deafened cochlear implant 
patients has been documented in a only few studies (21). 
Detailed work on the mechanisms of AV integration is 
an evolving field of research. To what degree and by 
what mechanisms do adult implant patients integrate the 
highly degraded speech signal presented via the cochlear 
implant? Goh and colleagues (20) reported an analysis 
of one adult cochlear implant patient's AV speech 
recognition. The patient was deafened at the age of 29 
and he underwent implantation with a Clarion device at 
age 31. At the time of the study, he had 4 years' expe- 
rience with the device and was an excellent performer 
based on City University of New York scores of 92% in 
the A condition. He was tested for understanding of 18 
sentences that were digitized and degraded to represent 
sine-wave speech. His scores in A, V, and AV conditions 
were compared with those of a group of normal-hearing 
subjects. In the AV condition, his score was 90%, which 
represented a gain of 78% by the addition of visual cues 
over the A condition (in which he scored 53%). Al- 
though in the A condition his scores were lower than 
normal-hearing subjects, his performance in the AV con- 
dition was comparable to the average performance of 
normal-hearing controls. They concluded that the audi- 
tory information provided by the implant was useful in 
multimodal speech, despite the absence of the speech 
cues traditionally thought of as critical elements of 
speech understanding. 

Giraud and Truy (22) performed positron emission 
tomography in postlingually deafened adult cochlear 
implant users while they listened to speech with their 
eyes closed. They observed activation of the visual cortex 
in both normal hearing and cochlear implant subjects. 
There was additional activation of the fusiform/perirhi- 
nal region (a region that is thought to invoke visual im- 
ages) and the midfusiform area (known as the face 



region) in cochlear implant subjects but not in controls. 
The authors concluded that higher visual cortical areas 
participate in processing of speech in cochlear implant 
subjects. Whether and to what degree patients with con- 
genital or early acquired deafness retain the ability to 
use multimodal speech is unknown. 

The subjects of this study would be considered poor 
candidates by many centers. They were congenitally 
deafened, had long-term deafness (between 18 and 45 
years), and had minimal residual hearing. However, in 
general, they were good lipreaders and had worn hearing 
aids for the majority of their lives. At the House Ear 
Clinic, these characteristics are thought to correlate with 
subjective satisfaction in prelingually deafened patients 
after implantation and are incorporated in the guidelines 
for preoperative selection and counseling. 

Despite poor performance on standard speech per- 
ception tests (Hearing In Noise Test, 0-83%; median, 
12.5%), the subjects in this study were all satisfied, full- 
time cochlear implant users. For all eight subjects as 
a group, the AV scores were significantly better than A 
or V scores. Three subjects appeared to have a simple 
additive effect in the AV condition. The remaining three 
subjects performed better in the AV condition than what 
would be estimated by simple addition of scores ob- 
tained in the A and V conditions (a superadditive effect). 

Two subjects did not show improvement in speech 
understanding in the AV condition compared with the 
unimodal conditions. One of these two subjects performed 
a few percentage points worse in the AV condition than 
in the V condition. His understanding of auditory signal 
was very poor, having scored less than 1% in the A 
condition. The degradation in AV speech understanding 
of only a few percentage points is probably insignificant. 
Despite this lack of benefit in objective testing, these 
two patients continued to be full-time users and reported 
satisfaction with their devices. This suggests that our 
understanding of benefit remains incomplete in this 
group of patients. Ongoing research will attempt to quan- 
tify subjective benefit. 

Overall, the results suggest a significant capacity for 
multimodal speech perception in congenitally deafened 
adult cochlear implant patients. AV speech perception is 
a complex system of central processing that incorporates 
transcription of auditory and visual features, integration 
of auditory and visual features, individual linguistic 
knowledge, individual memory and attention abilities, 
and understanding of context (23-25). For cochlear 
implant patients, especially those with little auditory 
experience, the implications of presenting a novel signal 
to the central auditory system cannot be underestimated. 
To what degree the novel signal can be integrated and 
whether it could cause cortical reorganization, interference 
of visual and auditory speech processing mechanisms, 
or overloading of the cognitive language capability is 
unknown. A better understanding of the mechanisms of 
AV speech in cochlear implant patients may guide ad- 
vancements in technology and direct postoperative 
rehabilitation. 
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CONCLUSION 

Some congenitally deafened subjects who undergo 
implantation as adults have substantial speech percep- 
tion benefit from the auditory signal. Despite scoring 
poorly in the auditory alone condition, there were signif- 
icant gains in speech perception when both auditory 
information from a cochlear implant and visual infor- 
mation by lipreading were available. This study shows 
that some congenitally deafened adults are able to in- 
tegrate auditory information provided by the cochlear 
implant (despite the lack of auditory speech experience 
before implantation) with visual speech information. 
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